Transmitting a video bitstream through wireless channels is a challenging problem due to limitations in bandwidth and a noisy channel. If a video is originally coded at a bit rate greater than an available bandwidth in a wireless channel, then the videos must first be transcoded to a lower bit rate, prior to transmission. Because a noisy channel can easily corrupt a quality of the video, there is also a need to make the encoded video bitstream resilient to transmission errors, even though the overall number of bits allocated to the bitstream is reduced.
Two primary methods used for error-resilience video encoding are resynchronization marker insertion and intra-block insertion (intra-refresh). Both methods are effective at localizing errors. If the errors are localized, then recovery from errors is facilitated.
Resynchronization inserts periodic markers so that when an error occurs, decoding can be restarted at a point where the last resynchronization marker was inserted. In this way, errors are spatially localized. There are two basic approaches for inserting synchronization markers: a group-of-block (GOB) based approach, which is adopted in the H.261/H.263 standard, and a packet-based approach, which is adopted in the MPEG-4 standard.
In the GOB-based approach, a GOB header is inserted periodically after a certain number of macroblocks (MBs). In the packet-based approach, header information is placed at the start of each packet. Because the way the packets are formed is based on the number of bits, the packet-based approach is generally more uniform than the GOB-based approach.
While resynchronization marker insertion is suitable to provide a spatial localization of errors, the insertion of intra MBs is used to provide a temporal localization of errors by decreasing the temporal dependency in the encoded video bitstream.
A number of error resilience video encoding methods are known. In “Error-resilient transcoding for video over wireless channels,” IEEE Journal on Selected Areas in Communications,” vol. 18, no. 6, pp. 1063-1074, 2000 by Reyes, et al., optimal bit allocation between error resilience insertion and video encoding is achieved by modeling the rate-distortion of error propagation due to channel errors. However, that method assumes that the actual rate-distortion characteristics of the video are known, which makes the optimization difficult to realize practically. Also, that method does not consider the impact of error concealment.
In “Optimal mode selection and synchronization for robust video communications over error-prone networks,” IEEE Journal on Selected Areas in Communications, vol. 18, no. 6, pp. 952-965, 2000 by Cote, et al., the optimal error resilience insertion problem is divided into two sub-problems: optimal mode selection for MBs; and optimal resynchronization marker insertion. That optimization is conducted on an MB basis and inter-frame dependency is not considered.
Another method described by Zhang, et al., “Video coding with optimal inter/intra-mode switching for packet loss resilience,” IEEE Journal on Selected Areas in Communications, vol. 18, no. 6, pp. 966-976, 2000, determines recursively a total decoder distortion with pixel-level precision to account for spatial and temporal error propagation in a packet loss environment. That method attempts to select an optimal MB encoding mode. That method is quite accurate on the MB level when compared with other methods. However, that method does not consider the inter-frame dependency and the optimization is only conducted on the current MB.
Dogan, et al. describe a video transcoding framework for general packet radio service (GPRS) in “Error-resilient video transcoding for robust inter-network communications using GPRS,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 6, pp. 453-464, 2002. However, the bit allocation between inserted error resilience and the video encoding is not optimized in that method.
For video distortion caused by channel errors, a low complexity video quality model has been described by Reibman et al., in “Low-complexity quality monitoring of MPEG-2 video in a network,” in Proceedings IEEE International Conference on Image Processing, September 2003. However, the measurement to determine error propagation effects is only based on the received bitstream. One of the most important aspects that is not fully considered by that method is the issue of inter-frame dependency, which is a key factor in motion compensated video encoding. Often, bit allocation and encoding mode selection are optimized only for the current MB or the current frame.
It is desired to provide an optimal solution that reduces the video bit rate while maintaining error resilience. It is also desirable to have models that account for inter-frame dependency, which is inherit to many coding schemes, and also accurately account for the propagation of errors at the receiver. This is especially important when a video bit stream is transferred from a channel with a high bandwidth and a low bit-error-rate (BER), for example, a wired channel, to a channel with a low bandwidth and a high BER, for example, a wireless channel. For such a low bandwidth channel, the combined task of bit rate reduction and error resilience insertion is essential because the bit rate reduction needs to be balanced against the additional error resilience bits.